Rare Outcomes, Common Treatments: Analytic Strategies Using Propensity Scores

Propensity Scores When treated patients are compared to controls, differing outcomes may reflect either effects caused by the treatment or differences in prognosis before treatment. Random assignment of patients to treatment or control, as in a randomized, controlled clinical trial (1), ensures that the groups were comparable before treatment and the prognosis in treated and control groups was nearly the same, so that differing outcomes indicate treatment effects. Somewhat more precisely, random assignment ensures that the only differences in prognosis between groups are due to chance, the flip of a coin in assigning treatments. In an ideal randomized trial, if a common statistical test rejects the hypothesis that the difference in outcomes is due to chance, a treatment effect is demonstrated. Notice that randomization does nothing to make patients have individually similar prognoses; rather, it ensures that assignment to treatment or control is unrelated to prognosis. When random assignment is not used—that is, in an observational study—treated and control groups may differ in prognosis, and differing outcomes may not be effects of the treatment. Measured and recorded differences in prognosis—overt biases—can often be controlled by analytical adjustments (2), whereas unmeasured differences—hidden biases—may exist and must be addressed by other means (2–4). A prognostic variable or covariate is a variable describing the condition of patients before treatment. Bias refers to systematic differences between treated and control groups with respect to one or more prognostic variables; the bias is overt if the variable is measured and hidden if it is not. Analytical adjustments for overt biases are of two kinds: 1) those that focus on the relationship between prognostic variables and outcomes and 2) those that focus on the relationship between prognostic variables and assignment of patients to treatment or control. The first strategy models the response directly, for example, through use of regression or logistic regression. The second strategy, which uses propensity scores, is an attempt to reconstruct, after the fact, a situation similar to random assignment, albeit only with respect to observed prognostic variables. In principle, either strategy (separately or in combination), properly used, can control overt biases. Neither strategy does much to control hidden biases. In practice, the second strategy has advantages over the first when the outcome is rare, the treatment is common, and there are many prognostic variables. Here, the terms rare and common refer to the available data: A rare outcome is seen in a small fraction of the patients under study, and thus there are limited data with which to model the outcome and its relationship to prognostic variables; however, a large fraction of patients received each of the two treatments under study, so there are plenty of data with which to model the relationship between treatment assignment and prognostic variables. An example of this can be seen in studies of relatively rare adverse side effects of relatively common treatments. One such study is by Jasmer and colleagues (5) in this issue: The authors examined 18 cases of hepatotoxicity among 411 patients given one of two competing treatments for latent tuberculosis infection.